When Paul has noticed this regression specific to Postgresql, we thought we may also run the tests with this database in github CI. Though it does not seem to be as easy as I first thought. We have understood from @berhalak comment that you already run server tests against this database in your internal CI. Is it possible you either open-source the tests, or share with us how you run them so we can add the same logic to the Github CI, if that makes sense to you?

Or do you think this work is not worth the effort?

Is it possible you either open-source the tests, or share with us how you run them so we can add the same logic to the Github CI, if that makes sense to you?

This is what we have:

RUN_gen_server_pg() {
  # Repeat some tests against postgres
  run with_postgres env TEST_CLEAN_DATABASE=true TEST_REDIS_URL=redis://localhost/{redis} TESTDIR=$OUTDIR/testdir \
    $BIN/mocha '_build/test/gen-server/**/*.js' '_build/core/test/gen-server/**/*.js' $MOCHA_FLAGS
}

So we repeat the gen-server tests against postgres. I don't see the server tests running against postgres. They may need modification to do so if they assume the database is a file.

These tests weren't enough to catch the IN () problem, which only showed up in a completely different battery of tests we apply to test the quality of an existing deployment.

Thanks @fflorent for your work! I've done some investigation on the failing tests. Most of the failures are timeouts due to the worse performance of Postgres (compared to SQLite). A significant number of other failures are simply chain effect.

I'm willing to take this forward if you'd like to hand it over to me. As @paulfitz discussed, the scope will be limited to gen-server for now, but within that scope I can see the potential of this PR getting merged. It should be fairly straightforward to fix the gen-server tests (but a decent amount of work to tweak timeout values), and I can later take a look at the nbrowser tests that are currently failing.

@SleepyLeslie I would be very pleased and grateful if you complete this work. Thank you very much for your investigation and your proposal to take this over!

As @paulfitz discussed, the scope will be limited to gen-server for now

Sounds very wise

Most of the failures are timeouts due to the worse performance of Postgres (compared to SQLite).

Something to do with the #773? Jordi fixed that with -c jit=off option for postgresql.
You may face an issue of Github actions that, IIRC, does not allow passing options to binaries in Docker image (so you might not disable the JIT compiler). If so, we might be able to circumvant the problem by using another (older?) version of postgresql not affected by the performance issues.

Something to do with the #773? Jordi fixed that with -c jit=off option for postgresql.

@fflorent Thanks for the pointer! I was able to run unmodified gen-server tests successfully with JIT turned off.

You may face an issue of Github actions that, IIRC, does not allow passing options to binaries in Docker image

I looked into that and found the environment variable POSTGRES_INITDB_ARGS. With its help I did a successful CI run with Postgres tests: https://github.com/gristlabs/grist-core/actions/runs/12205239155

I don't have write access to your fork so cannot push commits to your PR. I created a branch with my changes in our repo: https://github.com/gristlabs/grist-core/tree/add-postgresql-ci

Excellent news!

I don't have write access to your fork so cannot push commits to your PR. I created a branch with my changes in our repo: https://github.com/gristlabs/grist-core/tree/add-postgresql-ci

Please don't hesitate to open another PR. I'll close this one.

Closing in favor of #1336